Simultaneous Approximation with Neural Networks

In this paper we use a “uniformity” property of Riemann integration to obtain a single-hidden-layer neural network of fixed translates of a (not necessarily radial) basis function with a fixed “width” that approximates a (possibly infinite) set of target functions arbitrarily well in the supremum norm over a compact set. The conditions on the set of target functions are simple and intuitive: uniform boundedness and equicontinuity (so this result reduces to the “classical” theorems for a single target function). The uniformity property mentioned above refers to the existence of a single Riemann partition that achieves a prescribed accuracy of approximation of the Riemann integrals for a set of functions. A noteworthy feature of this simultaneous approximation scheme is that the nonlinear problem of finding the translates (also known as the “centers”) needs to be solved only once. The only parameters that need to be adapted for a particular target function are the weights from hidden-to-output layer (which is a linear problem).